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INTRODUCTION 


Psychologically-based  treatments  and  cognitive  behavioral  therapy  (CBT)  interventions  have 
been  shown  to  be  effective  in  alleviating  symptoms  of  Post-Traumatic  Stress  Disorder  (PTSD) 
and  related  psychological  health  problem  in  Veterans  and  military  personnel  who  suffer  from 
these  problems.  To  meet  the  increased  service  of  Veterans  with  PTSD,  new  training  methods 
need  to  be  developed  which  are:  1)  evidence-based,  and  2)  effective  in  modifying  and 
sustaining  changes  in  provider  behavior.  Methods  of  training/implementation  must  also  be 
scalable,  and  feasible  for  delivery  to  large  numbers  of  providers  in  cost-effective  ways. 
Internet-based  training  is  a  promising  new  approach  for  meeting  this  need,  but  has  received 
little  systematic  evaluation  to  date.  Noting  the  urgency  and  high  priority  of  this  issue,  Fairburn 
and  Cooper  (2011)  have  advocated  strongly  for  the  development  of  novel,  internet-based 
training  methods  and  innovative  research  designs  to  test  the  effectiveness  of  these  new  training 
methods.  Our  current  program  of  research  is  aimed  to  address  these  needs. 

The  broad  objective  of  our  research  is  to  design,  implement  and  evaluate  scalable  and  cost- 
effective  new  methods  for  training  of  mental  health  clinicians  providing  treatment  services  to 
veterans  with  PTSD.  The  randomized  controlled  trial  (RCT)  design  is  briefly  as  follows:  eligible 
clinicians  in  the  community  and  VHA  will  be  randomly  assigned  in  equal  numbers  to  three 
parallel  intervention  condition:  a)  Web-based  training  plus  web-centered  supervision;  b)  Web- 
based  training  alone;  and  c)  a  written  manual  control  group.  An  equal  number  of  clinician 
trainees  from  VHA  (N=219)  and  the  community  (N=219)  will  be  recruited  and  enrolled  in  the 
study  over  an  18-month  period  according  to  a  randomized,  stratified  24-week  design. 
Comprehensive  assessments  will  be  performed  at  baseline  (TO),  completion  of  training  (T1), 
and  at  3  month  follow-up  (T2).  Participants  randomized  to  the  consultation  condition  will  be 
exposed  to  a  newly  developed  web-centered  form  of  learning  consultation.  Measures  of 
compliance  and  completion  will  assess  adherence  to  protocol.  Training  effectiveness  will  be 
evaluated  by  means  of  a  combination  of  objective  (SPE)  and  self-report  measures. 

The  primary  and  secondary  aims  of  the  study  are  as  follows: 

Primary  Aim:  To  compare  an  enhanced,  internet-based  training  intervention  combined  with 
novel  web-centered  supervision,  internet-based  training  intervention  without  web-centered 
supervision  and  a  written  manual  control  with  regard  to  improvements  in  two  CBT-based  skill 
areas  (behavioral  task  assignment  and  case  conceptualization).  We  hypothesize  that  enhanced, 
internet-based  training  in  conjunction  with  web-centered  supervision  will  result  in  superior  skills 
acquisition  compared  to  internet  training  alone  and  that  internet  training  alone  will  result  in 
superior  CBT  skills  than  a  written  manual  control. 

Secondary  Aim  #1:  To  compare  improvements  in  knowledge  and  attitudes  following  internet- 
based  training  with  or  without  web-centered  supervision  and  the  written  manual  control.  We 
hypothesize  that  web-centered  supervision  will  lead  to  greater  improvements  in  CBT  knowledge 
and  perceived  self-efficacy  compared  to  internet-based  training  without  supervision  or  a  written 
manual  condition.  We  hypothesize  similarly  that  internet-based  training  will  be  associated  with 
improved  outcomes  in  CBT  knowledge  and  attitudes  compared  to  a  written  manual  control. 

Secondary  Aim  #2:  To  compare  improvements  in  skills  acquisition  in  knowledge  and  attitudes 
following  training  in  clinicians  recruited  from  VHA  mental  health  treatment  settings  compared  to 
those  providing  services  in  civilian  community-based  clinics.  We  hypothesize  that  comparable 
improvements  will  be  achieved  in  the  trainees  from  civilian  community-based  clinics  compared 
to  clinicians  recruited  from  VHA  centers. 

Secondary  Aim  #3:  To  determine  whether  clinician  implementation  of  skills  assessed  by 
means  of  a  novel,  objective  measure  of  skills  (i.e.,  standardized  patient).  We  hypothesize  that 


that  comparable  improvements  will  be  achieved  in  the  trainees  from  civilian  community-based 
clinics  compared  to  clinicians  recruited  from  VHA  centers. 

Secondary  Aim  #4:  To  assess  the  relative  efficiency  of  training,  as  measured  by  total  time 
required  for  training  in  each  condition,  in  addition  to  self-reported  level  of  burden  for  clinicians. 
We  hypothesize  that  internet-based  training  with  or  without  web-centered  supervision  will  be 
associated  with  increased  time  investment  and  burden  relative  to  training-as-usual,  but  that 
absolute  levels  of  burden  will  be  low  in  the  web  training  conditions. 

Our  study  will  be  the  first  of  its  kind  to  systematically  compare  web-based  training  interventions 
across  treatment  settings  and  provider  groups  (VHA  vs  Non-VHA).The  study  will  also  be  unique 
in:  1)  developing  and  testing  of  new  web-enhanced  training  modules  and  a  novel  web-centered 
supervision  model  recently  proposed  by  Fairburn  &  Cooper  (2011);  2)  development  and 
implementation  of  a  new  patient-reported  measure  of  clinician  skill  and  competency;  and  3) 
assessment  of  post-training  maintenance  of  skills  beyond  the  training  period.  Our  focus  on 
broad-based,  generic  CBT  skills  rather  than  more  narrowly  focused  protocol-based  skills  is 
another  innovative  aspect  of  our  proposed  study.  Finally,  the  use  of  standardized  patient 
methodology  for  assessing  outcomes  of  training,  and  planned  comparisons  with  self-report  and 
knowledge-based  assessment,  is  another  novel  feature  of  our  proposed  study. 

If  successful,  the  study  will  promote  a  better  standard  of  care  for  psychological  health  of 
Veterans  and  their  families  by  evaluating  technical  feasibility  of  two  training  models  in  evidence- 
based  skills  for  PTSD  treatment  providers  and  measuring  their  outcomes  and  effectiveness.  If 
successful,  the  study  will  provide  experimental  support  for  broad  implementation  of  these 
enhanced  new  training  methods  across  a  variety  of  treatment  settings. 


BODY 

This  section  describes  the  accomplishments  associated  with  each  task  outlined  in  the  approved 
Statement  of  Work  (SOW).  As  stated  in  our  SOW,  the  major  tasks  to  be  accomplished  in  year  1 
were: 

1.  Develop  and  Finalize  Study  Protocol  and  Measures 

A.  Eligibility  criteria,  exclusion  criteria,  screening  protocol 

i.  During  the  first  year  of  this  project,  the  team  finalized  the  eligibility  criteria,  exclusion 
criteria  and  screening  protocol. 

ii.  Screening  protocol  is  currently  being  prepared  for  programming. 

B.  Sample  frame,  web  contact  methods,  email  lists  identified 

i.  During  the  first  year  of  the  project,  the  team  has  secured  its  two  primary  recruitment 
strategies  (VA  networks  of  clinicians  and  recruitment  via  Give  An  Hour),  and  have 
also  identified  other  existing  email  lists  of  potential  participants,  including 
collaboration  with  VA  initiatives,  professional  organizations  and  community  based 
groups. 

a.  Email  lists  include:  VHA  mental  health  clinicians  that  have  participated  in  the 
“clinical  training  program”  at  the  National  Center  for  PTSD  in  the  past,  VHA 
mental  health  clinicians  who  have  not  participated  in  the  “clinical  training 
program”  at  the  NCPTSD,  team  leaders  at  the  Vet  Centers  in  the  United 
States,  and  a  registry  of  community  practitioners  who  have  agreed  to 
volunteering  pro  bono  to  serve  the  needs  of  veterans  (Give  an  Hour).  A  large 
number  (6800)  community  clinicians  participate  in  the  latter  registry  and  our 


study  has  arranged  access  to  this  recruitment  pool.  Additionally,  as  a  backup 
recruitment  strategy  we  have  compiled  a  listing  of  other  VA  organizations  and 
professional  organizations  that  can  assist  in  recruitment. 

b.  Postcards  have  been  created  to  be  disseminated  at  conferences  and  other 
mental  health  events  to  enhance  recruitment  activities  in  addition  to  the  pre¬ 
defined  email  lists. 

c.  lnformative/”Save  the  Date”  postcards  have  been  created  to  be  disseminated 
at  conferences,  other  mental  health  events,  and/or  to  eligible  clinicians  to 
enhance  recruitment  activities  and  allow  interested  individuals  the  opportunity 
to  pre-register  for  the  study  by  visiting  a  website  where  they  can  enter  their 
information  and  be  contacted  for  future  participation  with  the  training. 

C.  Standardized  Patient  Interviews  scripted  and  pre-tested 

i.  A  standardized  patient  interview  script  has  been  created  for  assessing  clinical  skills 
prior  to  and  following  the  training.  This  is  being  pre-tested  currently. 

ii.  A  transcription  service  has  been  identified  and  contracted  for  creating  transcriptions 
from  all  standardized  patient  interviews,  and  is  currently  providing  satisfactory  new 
pilot  transcripts  for  the  finalizing  of  our  second  rating  scale. 

iii.  Three  pilot  raters  have  been  hired  and  began  working  on  the  standardized  patient 
development  in  September,  2013. 

iv.  New  actors  and  raters  have  been  identified  for  the  study,  and  additional  personnel 
will  be  hired  in  the  upcoming  month. 

D.  Knowledge  and  attitude  questionnaires  have  been  developed  for  the  study,  which  are 
currently  undergoing  pre-testing.  Other  measures  are  being  finalized  based  on  findings 
from  the  first  study  (Ruzek  et  ai,  2012). 

E.  Consent  form  drafted,  human  subjects  protocol  finalized 

i.  During  the  first  year  of  this  project,  the  web  based  consent  form  was  created.  The 
consent  form  and  protocol  has  been  approved  by  both  Stanford  and  NERI  IRBs. 
The  terms  of  the  study  will  be  clearly  defined  with  an  “accept”  or  “decline” 
participation  button.  Those  who  accept  will  progress  to  the  next  phase  of  the  study. 
Those  who  decline  will  be  thanked  for  their  time.  All  consent  activities  will  occur 
through  a  secure  web  site. 

F.  VHA/NERI IRB  approvals  and  USAMRMC  HRPO  human  subject  protocol  approval 

i.  The  Protocol,  Recruitment  materials,  Informed  Consent  documents,  applicable 
applications  were  initially  approved  by  the  Stanford  IRB  (NCPTSD)  on  March  15th, 
2013  and  the  Research  and  Development  Committee  on  March  20,  2013.  Materials 
were  approved  by  and  New  England  Research  Institutes,  Inc  (NERI)  IRB  on  May  6, 
2013.  Materials  for  our  testing  of  knowledge  items  was  reviewed  and  approved  by 
the  Stanford  IRB  on  June  19th,  2014  and  the  NERI  IRB  on  June  20th,  2013  to  comply 
with  the  DoD  requirements. 

ii.  The  protocol  was  reviewed  by  the  US  Army  Medical  Research  and  Materiel 
Command  (USAMRMC),  Office  of  Research  Protections  (ORP),  Human  Research 
Protection  Office  (HRPO)  and  found  to  comply  with  applicable  DOD,  US  Army,  and 
USAMRMC  human  subjects  protection  requirements.  This  approval  was  dated  July 
2,2013. 


2.  Develop  and  Finalize  Web-based,  CBT  Training  Materials 

A.  Develop  and  pre-test  CBT  instructional  modules  and  materials 

i.  During  the  first  year,  NCPTSD  staff  co-led  the  content  development  modules.  All 
design  elements  have  been  developed  and  completed.  Module  content  for  the 
video  clip  development  has  been  finalized,  including  filming  for  videos  and  final  cut 
review.  The  BTA  web  interface  is  ready  for  use,  and  web  programming  of  content 
for  the  chain  analysis/case  formulation  module  is  on  schedule  for  completion  in 
October,  2013. 

B.  Develop  case  material  and  demonstrations 

i.  During  the  first  year  of  this  project,  the  team  developed  multiple  case  portrayals  for 
use  in  the  video  simulation  interviews.  All  video  simulation  interviews  are  complete 
and  are  being  programmed  into  the  chain  analysis  web-based  interface. 

C.  Prepare  web-based  supervision  manual  and  materials 

i.  During  the  first  year  of  this  project,  the  team  created  a  manual  and  materials  for  the 
web-based  supervision. 

ii.  This  manual  is  currently  being  reviewed  by  content  expert  Brett  Litz,  Ph.D.  for  final 
feedback. 

D.  Training  content  complete,  reviewed  by  CBT  expert  consultants 

i.  A  team  of  carefully  selected  expert  content  consultants,  including  Christopher 
Fairburn,  MD,  Amy  Naugle,  Ph.D.,  Gareth  Holmen,  Ph.D.,  and  Brett  T.  Litz,  Ph.D., 
was  assembled.  These  individuals  were  invited  to  serve  as  special  consultants  to 
the  project  based  upon  their  extensive  experience  and  knowledge  in  these  content 
areas. 

E.  Pilot  test  all  study  procedures  and  materials  (prior  to  programming) 

i.  Pilot  testing  of  the  knowledge  items  is  in  its  final  stages.  Knowledge  items  were 
developed  this  year  through  a  combination  of  blueprinting  and  concept  mapping 
approaches.  This  process  identified  key  concepts  and  to  generate  specific 
knowledge  assessment  items.  Team  and  expert  consultant  review  resulted  in  a  final 
set  of  50  pilot  items.  An  online  survey  was  programmed  for  the  piloting  of  these 
items,  and  110  surveys  were  collected  for  three  target  groups:  VA  clinicians, 
community  practitioners,  and  cognitive  behavioral  therapy  experts.  The  surveys 
collected  both  qualitative  and  quantitative  data.  This  dataset  is  currently  being 
cleaned  and  analyzed  according  to  our  study  protocol.  Final  items  will  be  available 
at  randomization. 


3.  Develop.  Pre-Test  and  Finalize  Web  Site  and  Instructional  Program 

A.  VHA  web  host  programmers  provide  specification  and  guidance  to  web  programmers 
and  database  programmers 

i.  The  team  has  collaborated  with  NERI  to  determine  integration  requirements  for  in 
the  future  hosting  the  web-based  training  program  on  an  NCPTSD  server.  There 
are  no  anticipated  issues  with  hosting  the  program  with  the  National  Center  for 
PTSD.  The  site  will  be  compliant  with  and  fully  tested  for  508  compliance 


requirements. 


B.  PTSD  training  material  completed.  All  web  training  modules  have  been  completed  and 
are  being  beta  tested  currently. 

C.  Web  programming  specifications  completed 

i.  During  the  first  year  of  this  project,  the  web  programming  specifications  were 
identified,  content  was  finalized  and  web  programming  activities  were  initiated.  Web 
programming  including  all  interactive  elements  is  on  schedule  for  completion  in 
October. 

D.  Web  program  transferred  to  VHA  intranet  server 

i.  Web  programming  has  not  yet  been  transferred  to  the  VHA  server  as  initially 
planned.  This  site  will  be  hosted  externally  until  the  study  recruitment  is  completed, 
at  which  time  it  will  be  loaded  and  tested  on  the  VHA  NCPTSD  server. 


4.  Develop  Data  Management  System 

A.  Flow  chart  all  study  steps,  web  data  collection  and  database  requirements 

i.  During  the  first  year  of  this  project,  a  design  was  created  to  show  the  steps  in  data 
collection  to  plan  for  programming  of  the  DM  system. 

B.  Develop  web-based  data  forms  for  all  research  measures 

i.  During  the  first  year  of  this  project,  the  team  has  begun  the  creation  of  web-based 
data  forms. 

C.  Program  the  database  including  project  data  entry  forms 

i.  During  this  period,  the  team  conducted  initial  programming  activities  of  forms  in 
preparation  for  the  pilot.  These  activities  will  be  ongoing  over  the  next  two  months, 
with  completion  by  launch  in  the  first  quarter  of  the  second  year  grant  cycle. 

D.  Program  web  usage  statistics  reports 

i.  An  analysis  plan  has  been  developed  for  selection  and  validation  of  knowledge 
items.  These  reports  will  be  beta  tested  prior  to  initiation  of  the  study. 

E.  Program  reports  to  monitor  project  operations  and  outcomes 

i.  The  project’s  team  members  at  NERI  will  be  using  an  electronic  case  report  from 
systems,  eCOS,  to  store  the  data  forms.  The  reports  will  be  created  to  assist  with 
monitoring  project  operations.  Plans  have  been  initiated  in  this  project  year,  and 
implementation  will  be  undertaken  in  the  next  project  year. 


5.  Major  Task  Milestone 

A.  Program  automatic  email  reminders/interaction  with  web  participants 

i.  During  the  first  year  of  this  project,  the  email  reminder  process  has  been  reviewed 
and  is  an  ongoing  part  of  the  data  management  activities. 

B.  Beta  test  all  programmed  pieces  and  interactivity  with  web  course 

i.  Beta  testing  will  be  completed  by  launch,  the  first  quarter  of  the  second  year  grant 


cycle.  This  will  be  conducted  to  ensure  that  there  are  no  interactivity  errors  with  the 
web  course  of  the  evaluation  materials. 


6.  Prepare  Standardized  Patient  Rating  Protocol 

A.  Develop  and  Pre-test  Standardized  Patient  (SP)  Rating  Guide 

i.  Standardized  patient  case  materials  have  been  developed  and  are  in  pre-testing 
phase. 

ii.  New  rating  scales  have  been  developed  and  are  being  pre-tested  currently.  Rating 
scales  are  currently  undergoing  rigorous  review  and  improvement  for  clarity  of 
language,  structure,  and  inter  rater  reliability. 

B.  Study  raters  and  actors  have  been  engaged  for  the  study 

Pilot  raters  have  been  hired  and  rating  scales  are  currently  undergoing  rigorous 
review  and  improvement  for  clarity  of  language,  structure,  and  inter  rater 
reliability.  Main  study  rater  and  actor  positions  have  been  posted  and  are 
currently  being  reviewed  and  interviewed. 

C.  Training  procedures  have  been  developed  and  pre-tested. 

Full  drafts  of  training  materials  have  been  created  and  will  be  finalized  once  the 
rating  scales  are  final. 

KEY  RESEARCH  ACCOMPLISHMENTS 

•  Hiring  and  training  of  project  staff 

•  Establishment  of  ongoing  communication  system  and  decision-making  processes  with 
partnering  institution  NERI 

•  Prepared  and  finalized  research  protocol 

•  Draft  knowledge  and  skills  assessment  measures  and  initiate  pilot  testing 

•  Obtained  all  relevant  approvals  for  study  protocol,  informed  consent  forms,  and 
recruitment  materials 

•  Developed  online  training  materials 

•  Provided  expert  feedback  and  input  on  web  programming  activities 

•  Authored  content  on  web  programming  activities 

•  Hired  three  new  part-time  Standardized  Patient  Raters 

•  Trained  three  new  part-time  Standardized  Patient  Raters 

•  Conducted  Initial  rating  sessions  with  Standardized  Patient  team  to  develop  rating 
scales 

•  Setup  contract  with  Viva  Transcription,  a  voice-to-text  transcription  service 

•  Ordered  and  inventoried  audio  and  recording  equipment  for  standardized  patient  and 
clinician  phone  calls 

•  Developed  and  setup  scheduling  system  for  standardized  patients  and  participants  to 
utilize 


•  Provided  therapy  expertise  and  acting  resources  for  video  filming  project 

•  Developed  study  assessment  methods  and  materials 

•  Pre-testing  of  study  assessment  methods  currently  underway 

•  Pilot  testing  was  initiated 


During  the  next  performance  period,  Year  2,  the  team  will  focus  on  the  following  activities: 

1.  Initiate  Recruitment  Activities  and  Secure  Adequate  Enrollment:  Based  on  our 
established  plan  for  recruitment,  the  team  will  develop  a  data  system  for  screening, 
consenting  and  tracking  study  participants.  We  will  initiate  recruitment  and  in 
subsequent  months  adjust  our  efforts  based  on  rates  of  enrollment,  such  that  we  will 
have  consistent  enrollment  during  the  period  in  which  participants  will  be  in  enrolled  (i.e., 
the  first  18  months  of  the  study). 

2.  Develop  Data  Management  System:  The  team  will  continue  to  work  on  development  of 
web  based  data  forms  for  all  research  measures  and  programming  the  database  to 
include  project  data  entry  forms. 

3.  Launch  of  web-based  training  program:  Completion  of  web  programming  for  training 
program,  including  installation  of  all  video  elements  (currently  being  reviewed  and 
undergoing  final  edit).  Launch  of  site  for  study  is  planned  for  the  first  quarter  of  the 
second  year  grant  cycle. 

4.  Complete  planning  and  implementation  of  email  reminders/interaction  with  web 
participants:  Email  reminders  are  anticipated  to  be  programmed  by,  the  first  quarter  of 
the  second  year  grant  cycle. 

5.  Standardized  Patient  Rating  Scales:  Pre-testing  of  interviews  and  standardized  patient 
rating  scales  are  in  process  and  scheduled  for  completion  in  Q5,  2013. 

6.  CE/CME  Accreditation:  As  study  participants  cannot  be  compensated  for  their  time  with 
anything  of  monetary  value,  CE/CMEs  will  be  issued  for  completion  of  the  web-based 
training.  CE/CME  accreditation  and  related  activities  are  in  process. 


To  date  there  have  been  no  risks  or  unanticipated  issues  associated  with  this  project  that  have 
impeded  its  performance. 


REPORTABLE  OUTCOMES 

To  date,  there  are  no  reportable  outcomes. 


CONCLUSION 

This  training  program  will  focus  on  development  of  evidence-based  CBT  skills  to  improve  skills 
in  providers  treating  Veterans  and  active  duty  military  and  to  effectively  engage  patients  in  the 
treatment  process.  This  innovative  study  will  add  new  knowledge  to  our  understanding  of  skills 
dissemination  in  PTSD  provider  care.  We  will  test  the  hypotheses  of  the  study  in  a  rigorous, 
experimental  design,  and  will  assess  outcomes  of  new  web-based  training  modules  and 
consultation  methods.  This  study  will  provide  data  to  assist  researchers,  military  leaders,  and 


treatment  providers  to  better  understand  practical  and  theoretical  implications  for  future  training 
of  mental  health  providers  in  the  VAH  and  other  health  systems. 

All  major  milestones  were  met  in  the  first  12  months  of  grant  activity. 
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